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ABSTRACT 



The overall purpose of this evaluation study was to 
determine the effectiveness of providing 5th and 6th grade students in Walled 
Lake Consolidated Schools (WLCS) (Tennessee) with access to laptop computers 
with regard to classroom learning activities, technology usage, and writing 
achievement. The Laptop Program classrooms were equipped with wireless access 
to the Internet and printers. The program also provided students and parents 
the opportunity to receive training on basic computer skills. The evaluation 
was structured around three primary research questions: (1) Is teaching 

different in a Laptop classroom? (2) Do students behave differently in a 
Laptop classroom? and (3) Do students achieve differently in a Laptop 
classroom? Data were gathered through classroom observations, student writing 
test scores, student surveys and focus groups, teacher surveys and 
interviews, and parent surveys and interviews. In this evaluation of the 
first year of the Laptop Program, the results were consistently supportive of 
beneficial impacts on students, teachers, and parents. Specifically, all 
three groups believed that the program was positively changing teaching and 
learning both at school and at home. (MES) 
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Introduction 

The overall purpose of this evaluation study was to determine the effectiveness of providing 5th 
and 6th grade students in Walled Lake Consolidated Schools (WLCS) with access to laptop 
computers with regard to classroom learning activities, technology usage, and writing achievement. 

The WLCS Laptop Program is based on the Anytime Anywhere Learning (AAL) program (AAL, 
2000), which has been in schools since 1996 and has impacted more than 100,000 students and 
teachers. The goal of the AAL program is to provide students the knowledge, skills and tools to 
learn anytime and anywhere. 

The Laptop Program arranged to have laptop computers available for a monthly lease fee of fifty 
dollars. The Laptop classrooms were equipped with wireless access to the Internet and printers. 

The program also provided students and parents the opportunity to receive training on basic 
computer skills. The Laptop teachers received ten full days of professional development prior to 
the 1999-2000 academic year and six one-half day sessions during the year. The training was based 
on the NTeQ model (Morrison, Lowther, & DeMuelle, 1999) which provides teachers a 
framework to develop problem-based lessons that utilize real-world resources, student 
collaboration, and the use of computer tools to reach solutions. The lessons are typically structured 
around projects, which engage the students in critically examining community and global issues, 
while strengthening student research and writing skills. 



Research Questions 

The evaluation of the Laptop Program was structured around three primary research questions that 
focused on classroom practices, student behavior and writing ability. The detailed questions are 
listed below! 

• Is teaching different in a Laptop classroom? To answer this question, observers 
examined classroom practices to determine if instructional practices in Laptop 
classrooms were different from those in non-Laptop classes. For example, were 
classrooms lecture-based and/or project-based, were the classrooms academically focused 
and were students engaged, did teacher questions call for students to construct responses 
or simply recall factual information. 
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• Do students behave differently in a Laptop classroom? By observing and talking to 
students, observers gauged the level of interest in learning, student attitude toward one 
another (do they get along and are they helpful), and the degree to which students take 
initiative for their learning as opposed to being dependent on the teacher for constant 
direction. 

• Do students achieve differently in a Laptop classroom? Observers assessed writing 
samples from Laptop and non-Laptop classrooms looking for both content and quality, 
observed whether writing in the classroom was sustained or short-term question and 
answer, and whether technology was used as a tool to increase the quality of work or 
simply for computer assisted instruction. 



Design 

The evaluation period extended from September 1, 1999 through May 30, 2000. The evaluation 
design was based on both quantitative and qualitative data collected from students, teachers, and 
parents involved with the Laptop Program and students and teachers in non-Laptop classrooms in 
seven schools (four elementary and three middle) within WLCS. Comparative analyses were 
completed for teaching activities and learning outcomes and descriptive analyses were completed 
for student, teacher, and parent reactions to the Laptop Program. 

The data set for the evaluation included classroom observations, student writing test scores, 
student surveys and focus groups, teacher surveys and interviews, and parent surveys and 
interviews. Two separate observation measures were used to collect observation data*. The School 
Observation Measure (SOAlf), and the Survey of Computer Use ( SCU ) . SOM was based on 60 
continuous minutes of observation, divided into about 4, 15-minute segments. These 4 
observation periods were then summarized on one SOM Data Summary form. SCU was 
completed as part of the 60-minute observation sessions, only if students used technology during 
that time. A total of 50 classroom observations were conducted, with 32 in Laptop classrooms and 
18 in non-Laptop classrooms. 

The WLCS’s Writing Scoring Guide was used to assess prompted writing samples from Laptop and 
non-Laptop students. A sample of 32 Laptop and 32 non-Laptop students were randomly selected 
to complete the writing test. Experienced reviewers used the district’s four-point rubric (ranging 
from 1 to 4, with 4 being the highest rating possible) to conduct a blind assessment of the writing 
samples for Ideas and Content, Organization and Form, Style, and Conventions, yielding four 
scores per student. 

The student, teacher, and parent surveys, interviews, and focus groups primarily focused on three 
areas: have the laptop computers had a personal impact (increased skills - research, computer, 
learning), have the laptops impacted what happens in the classroom, and what are the benefits, 
difficulties, and ways to improve the program. The final data set includes: 397 student surveys, 58 
student participants in focus groups, 13 teacher surveys, 7 teacher interviews, 187 parent surveys, 
and 40 parent interviews. 



Results 

Classroom Observations 

SOM® — As indicated in the description of SOM®, the observation procedure focused on 24 
instructional strategies using a five-point rubric (0 = not observed, 1 = rarely, 2 = occasionally, 3 = 
frequently, and 4 = extensively). Two additional items use a three-point scale (1 = low, 2 = 
moderate, 3 = high) to rate the degree to which academically-focused class time and student 
attention/interest/engagement are evidenced. In an initial analysis of the SOM® data, rubric 
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categories 2-4 were collapsed into one category to yield a two-category scheme reflecting the 
percentage of visits in which a strategy was either observed or not observed. As seen in Table 1, the 
analysis revealed significant differences, which favored Laptop over the Control teachers on 
project-based learning (65% observed vs. 22%), independent inquiry/research (58% vs. 24%) 
computer for instructional delivery (22% vs. 0%), and computer as a learning tool (88% vs. 17%). 
In general, strategies promoting learner activity, such as cooperative learning, inquiry, sustained 
writing, and computer uses were more likely to be observed in Laptop classrooms. 



Table 1: Proportion of times an event was observed (1-4) versus not observed (0) 



Strategies 




Laptop 


Control 


Observed 


Not observed 


Observed 


Not observed 


Direct instruction 


68.8 


31.3 


77.8 


22.2 


Team teaching 


15.6 


84.4 


11.1 


88.9 


Cooperative learning 


65.6 


34.4 


38.9 


61.1 


Individual tutoring 


13.3 


86.7 


11.1 


88.9 


Ability groups 


0.0 


100.0 


0.0 


100.0 


Multi-age grouping 


0.0 


100.0 


0.0 


100.0 


Work centers 


3.1 


96.9 


11.1 


88.9 


Higher level instructional feedback 


61.3 


38.7 


38.9 


61.1 


Integration of subject areas 


21.9 


78.1 


5.6 


94.4 


Project-based learning** 


64.5 


35.5 


22.2 


77.8 


Use of higher-level questioning 


56.3 


43.8 


50.0 


50.0 


Teacher as facilitator 


71.9 


28.1 


61.1 


38.9 


Parent/community involvement 


0.0 


100.0 


5.6 


94.4 


Independent seatwork 


71.9 


28.1 


55.6 


44.4 


Hands-on learning 


19.4 


80.6 


16.7 


83.3 


Systematic individual instruction 


0.0 


100.0 


5.9 


94.1 


Sustained writing/composition 


53.1 


46.9 


38.9 


61.1 


Sustained reading 


28.1 


71.9 


38.9 


61.1 


Independent inquiry/research* 


58.1 


41.9 


23.5 


76.5 


Student discussion 


50.0 


50.0 


44.4 


55.6 


Computer for instructional delivery* 


21.9 


78.1 


0.0 


100.0 


Computer as a tool** 


87.5 


12.5 


16.7 


83.3 


Performance assessment 


37.5 


62.5 


22.2 


77.8 


Student self-assessment 


18.8 


81.3 


16.7 


83.3 



*p< .05; **p< .01; ***p< .001 



There were seven comparisons that yielded statistically significant differences from t-tests 
comparing the means for Laptop and Control classes on each SOM item, all of which had 
associated effects sizes of .59 or higher in absolute value (see Table 2). All of the significant 
differences favored the Laptop classes', computer as a learning tool (ES= +2.29), project-based 
learning (ES= +0.95), independent inquiry (ES= +0.89), higher-level instructional feedback ( ES = 
+0.61), teacher as facilitator (ES= +0.64), cooperative learning ( ES= +0.59), and computer for 
instructional delivery {ES= +0.59). 
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Table 2: A Summary of Items Showing Significant Differences Between Laptop and Control Group 
Comparisons on the SOM®* 



Items Using Rating Scale A** 


Laptop 


Control 




P 


ES 


M 


SD 


M 


SD 


t 


Computer used as a tool 


2.84 


1.43 


.16 


.38 


7.71 


.000 


2.29 


Project-based learning 


2.25 


1.84 


.66 


1.32 


3.21 


.002 


0.95 


Independent Inquiry 


1.90 


1.81 


.52 


1.12 


2.83 


.007 


0.89 


Higher-level instructional feedback 


1.64 


1.53 


.77 


1.16 


2.07 


.044 


0.61 


Teacher as facilitator 


2.40 


1.70 


1.38 


1.37 


2.17 


.035 


0.64 


Cooperative learning 


1.71 


1.50 


.88 


1.18 


2.01 


.050 


0.59 


Computer for instructional delivery 


.65 


1.35 


.00 


.00 


2.04 


.047 


0.59 



*Sorted by Effect Size 

**Rating Scale A 

0 = Not Observed 

1 = Rarely Observed 

2 = Occasionally Observed 

3 = Frequently Observed 

4 = Extensively Observed 



SCU — Laptop classes, as would be expected, contained more computers (p< .001) than did 
Control classes. Additional areas where significant differences occurred were that Laptop classes 
had more! (a) PC’s, (b) up-to-date computers, (c) Internet access, (d) printer access, (e) color 
printer access, (f) computers clustered together, and (g) computers that were distributed. Further, 
Laptop classes always had at least one student at one computer and rarely had more. By 
comparison, about half of the Control classes averaged one student per computer, while half had 
more than five students per computer. All three comparisons involving the availability of 
computers to students significantly favored the Laptop classes. With regard to student technology 
skills, Laptop students were rated significantly higher than were Control students on computer 
skills (p< .001), keyboarding skills (p< .001), and mouse skills (p< .01). 

Comparisons of observation means using t-tests revealed statistically significant differences, most of 
which are noted above, and collectively show that Laptop classes provided greater access to 
computers and associated peripheral equipment to develop higher skill levels by students, to engage 
students and teachers more extensively in computer applications, to use computers more for 
research and for production in writing and design, and to make greater use of word-processing and 
Internet software (see Table 3), Importantly, on the final rubric, Laptop classes were rated as 
making much more meaningful use of computers compared to Control classes {Ms = 3.18 vs. 

LOO, ES= +2.72). 
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Table 9: Computer Impact 


Group 




Not Observed 


Rarely 


Occasionally 


Frequently 


Extensively 


Laptop 


Computer (s) worked 
well*** 


4.0% 


0.0% 


0.0% 


8.0% 


88.0% 




Students were very engaged 
in computer activities*** 


8.3% 


8.3% 


0.0% 


20.8% 


62.5% 




Teacher provided technical 
coaching** 


18.2% 


13.6% 


4.5% 


13.6% 


50.0% 



Control Computer (s) worked 


83.3% 


0.0% 


0.0% 


8.3% 


8.3% 


well*** 












Students were very engaged 
in computer activities*** 


83.3% 


0.0% 


0.0% 


16.7% 


0.0% 


Teacher provided technical 
coaching** 


81.8% 


0.0% 


0.0% 


18.2% 


0.0% 



*p<.05, *><.001, **><.001 

Writing Performance 

Writing Scores. Students in Laptop (/?= 32) and Control (n= 32) classes were asked to write a 
prompted essay. The essays were then scored in the blind on a rubric encompassing the four 
dimensions of Organization, Idea, Style, and Conventions. For each dimension, the essay was 
scored from 1 to 4, with 4 being the highest rating possible. 

Mean performance scores for Laptop and Control students were analyzed via a one-way 
multivariate analysis of variance (MANOVA) with the four dimension scores serving as the 
dependent variables. The MANOVA yielded a significant program effect (p= .048), therefore, 
univariate analysis of variance (ANOVA) was performed separately on each dimension. All four 
tests were highly significant and indicative of higher performance by Laptop than Control 
students. Effect sizes ranged from +0.61 to +0.78, suggesting moderately strong and educationally 
important effects. 



Student Reactions 

Student Survey. The Laptop student survey responses (n = 397), indicated that students felt their 
computer skills had increased, and they were better able to do Internet research. They were less 
certain that using computers at school increased their interest in learning, made them want to get 
better grades, improved their writing, or made it easier for them to work with other students. Over 
half of the students reported fairly regular use of the laptop and the Internet for completing 
homework, while even more reported uses for “other things.” The two most frequently cited “other 
things” were e-mail/chat and games.e-mail. 

When students were asked to describe the best thing about having a laptop, students included that 
it helped them learn computer skills, helped with school assignments, provided access to the 
Internet, and it helped the students become more organized. When students were asked about the 
hardest part of having the laptop, there was general consensus that it was difficult to keep track of 
and carry back and forth to school. Other concerns included reoccurring technical problems (e.g., 
freezes, charging, slow), using Microsoft Access, and students lacking sufficient computer skills. 
Overall, the survey results show that Laptop students were highly appreciative of having laptop 
computers and were taking advantage of its resources for performing a variety of learning activities 
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both at school and at home. Students were more likely to experience benefits of the laptop 
activities for the development of specific technology skills than for increasing their basic interest in 
school and grades. 

Student Focus Group. The researchers conducted six student focus groups that involved a total of 
58 students. Results from the focus groups closely align with findings from the student survey. 
When looking at learning and performance, the students indicated that they were more involved in 
writing, researching, and in collaborative project work. Many students reported an improvement in 
grades, although some students indicated there was no change in grades, and a few said some 
grades had dropped. Students felt they had a closer relationship with their teachers, more self 
confidence, and improved attitudes towards school. 

The majority of the students indicated that their parents liked the Laptop program. Others felt the 
Laptop program had improved relationships between students and their parents, that parents 
provided more help with homework, and that parents were amazed/happy at how quickly students 
had gained computer skills. All of the final comments were positive and indicated that the students 
liked having the laptop and looked forward to using it again next year. 



Teacher Reactions 

Teacher Survey. Thirteen Laptop teachers responded to the Teacher Survey. Results indicated that 
teachers were extremely positive regarding the benefits of the Laptop Program for them and their 
students. All agreed that the program experience: (a) increased their basic skills in computer 
applications, (b) increased the emphasis on higher-order learning in their classroom, (c) increased 
project-based learning, and (d) was beneficial to them as teachers. There was also strong agreement 
that they! were better prepared to create lessons integrating computers, frequently integrated 
technology, school- related interactions with students and parents increased, and would like to 
participate in the project again next year. 

The teachers indicated that the greatest benefit of the Laptop program was for students to have 
access to technology and Internet resources. The teachers also felt that use of the laptop had 
resulted in students having greater research skills, improved writing skills, interest in school, and 
greater self-confidence. The difficulties cited were all related to the technology itself, e.g., power, 
weight, drives, server, and printers. They were also concerned with students tampering with 
software and the laptop settings. As could be expected, teachers indicated that the program could 
be improved by providing more technical support, more basic training, providing a solution to the 
power problems and providing more projectors. 

Teacher Interviews. There were seven randomly selected Laptop teachers who were interviewed. 
Teachers indicated that classroom practices had changed due to the laptops in that they used more 
cooperative learning, completed more projects, and acted as facilitators of learning more 
frequently. Teachers reported that the projects involved more integration of subjects, research, 
higher-levels of learning, writing, and the use of spreadsheets, word processing, and the Internet 
than non-laptop projects. The teachers reported that they use authentic assessment and involve 
students in self-assessment and the development of rubrics now more frequently. As a result, 
teachers indicated that students produce higher quality work and had more self-confidence, greater 
enthusiasm, increased depth of knowledge, and were more engaged with other learners. Teachers 
indicated that there were fewer missed assignments and an overall improvement in grades. 



Parent Reactions 

Parent Survey. Encouragingly, parents (n= 187) generally viewed the Laptop Program as helpful 
to their children’s education. More than half felt that the program increased their child’s interest in 
school, involvement in project-type school work, and research skills. Between one-third to one-half 
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believed that increases occurred in school achievement, writing skills, and ability to work with 
other students. 

Results from the open-ended items show that over one-half of the parents stated that the most 
beneficial part of the Laptop program was that their child had improved his/her knowledge in 
different subject areas and also improved in computer literacy. The parents expressed concerns that 
it was difficult for their child to keep track of, be responsible for, and carry the laptop to and from 
school. Other concerns were related to monitoring student use of the Internet and overuse of 
computer games. The parents felt that more training is needed for teachers, parents and students 
(keyboarding). Another suggestion was to offer the program to all students in the district. 

Parent Interview. The parent interviews were conducted with a random selection of 40 parents 
(20 5 th grade, 20 6 th grade) whose children were participating in the Laptop study. Overall, the 
parents were supportive of the Laptop Program and felt that it has had a positive impact on the 
child’s learning and participation in school. There was a general consensus that the Laptop 
Program was providing their child with important computer, organizational, and research skills 
that are of benefit now and will enhance their future work opportunities. Most of the parents 
indicated that the laptop had little influence on the family, however, a few noted positive impacts 
on younger siblings. The majority of the parents also reported that the laptop had not changed 
interactions with the child or teacher primarily because they were already actively involved. 



Discussion 

Results of this study suggest varied impacts of the Laptop Program on students, teachers, and 
family members. These findings are discussed below in reference to the three primary research 
questions. 



Is Teaching Different in a Laptop Classroom? 

According to both teacher reports and classroom observations, Laptop classes are being taught 
differently than regular (Control) classes. Not only did the former classes incorporate technology 
to a much greater degree, they tended to employ more student-centered strategies such as project- 
based learning, independent inquiry/research, teacher as coach/facilitator, and cooperative learning. 
Most revealing in the study were the ways in which technology was accessed and employed in the 
Laptop classrooms. Compared to their Control counterparts, the Laptop students demonstrated 
more technical skill with computers and used computers more extensively for a variety of 
production and research functions. Not surprisingly, observers rated Laptop classes as making 
much more meaningful usage of computers as educational tools. 

Nearly all teachers believed that they were teaching differently than before by integrating 
technology into both newly developed lessons and existing lessons that had previously been taught 
without computers. Further, nearly all felt that they had increased the frequency of project-based 
learning, higher-order learning activity, and school-related interactions with parents and students. 
Laptop parents reported that their child was taking advantage of the laptop computer for school 
and other activities, especially in developing research skills. 

The implication from these multiple data sources is that teaching and learning were being 
impacted, in ways that promoted active learning and technology applications, as a consequence of 
all students having continual access to individual computers. Not surprisingly, although 
cooperative learning was observed relatively frequently in Laptop classes, students typically worked 
individually while using computers. Thus, they benefited from having their own computer to 
complete their work, while still being able to collaborate easily with others on information and 
strategies. 
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Do Students Behave Differently in a Laptop Classroom? 

As described above, Laptop students were more active, autonomous, and collaborative in their 
classroom behaviors. For example, cooperative learning was observed “frequently” or “extensively” 
in 35% of the Laptop classes, but only 11% of the Control classes. Students frequently or 
extensively engaged in projects in 55% of the Laptop classes compared to only 17% of the Control 
classes. Laptop teachers confirmed these impressions by describing their students as more 
independent, active, and engaged. The teachers were highly impressed with students’ abilities and 
interests in using computers to enhance learning. 

In their survey and interview responses, students indicated they had increased their computer skills 
substantially and were much more prepared to do Internet research. About two-thirds of the 
students generally worked with the laptop alone in the classroom, but they still collaborated 
frequently with others in sharing information, asking questions, and providing assistance. As a 
group, the students were less committal about the effects of the laptop in increasing the interest in 
learning, writing skills, and facilitating collaboration, although about one-third (still a substantial 
number) felt that they did realize these types of benefits. 



Do Students Achieve Differently in a Laptop Classroom? 

In this study, we assessed student achievement in terms of writing performance on a prompted 
essay. Grading, using a four-point rubric, was “blind” to students’ enrollment in Laptop vs. 
Control classes. Results significantly favored the Laptop group on all evaluation 
dimensions — Organization, Ideas, Style, and Conventions. Aside from being statistically 
significant, the differences across all dimensions reflected relatively strong advantages for the 
Laptop group, with effect sizes ranging from +0.61 to +0.78. 



Conclusions 

In this evaluation of the first year of the Laptop Program, the results are consistently supportive of 
beneficial impacts on students, teachers, and parents. Specifically, all three groups believed that the 
program was positively changing teaching and learning both at school and at home. These 
impressions were directly confirmed in visits to Laptop versus Control classrooms. While more 
research is needed on how the Laptop Program impacts student achievement, the positive results 
from the writing assessment are highly suggestive. Laptop students were doing more sustained 
writing in class and were demonstrating more skill in writing, making a causal connection highly 
likely. Control classes could also increase their emphasis on writing, but it is obvious that continual 
and immediate access to computers provided the Laptop students and their teachers with a very 
strong advantage. In future research, we hope to examine whether Laptop students demonstrate 
comparable advantages in problem solving. We anticipate that they will, given the extensive project 
and inquiry activities in which they engage. At this point, given the present data, we are most 
certain of one program result — Laptop students are much more fluent than other students with 
using the technology of the 21 st Century for learning, research, and production. For them, 
computers are fully integrated with and a natural part of their educational experiences both at 
school and at home. 
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